Cognitive machine learning classifier generation

ABSTRACT

A computer-implemented method includes creating a classifier by: training a machine learning model using two or more tasks, wherein the tasks lie in two or more domains; including in the machine learning model at least one attribute common to at least two of said two or more domains; including in the machine learning model at least one latent feature that affects at least two of the two or more tasks that fall within one of the at least two domains; and constructing the classifier based on said machine learning model. The computer-implemented method further includes applying the classifier to at least one operational task.

BACKGROUND

The present invention relates generally to the field of machinelearning, and more particularly to generating classifiers from cognitivemachine learning models.

“In the field of machine learning, Latent Dirichlet Allocation (LDA) isa generative statistical model that allows sets of observations to beexplained by unobserved groups that explain why some parts of the dataare similar.” Latent Dirichlet Allocation, WIKIPEDIA, THE FREEENCYCLOPEDIA, https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation(last accessed May 3, 2016). “Labeled LDA [is] a topic model thatconstrains [LDA] by defining a one-to-one correspondence between LDA'slatent topics and user tags.” Daniel Ramage et al., Labeled LDA: Asupervised topic model for credit attribution in multi-labeled corpora,1 EMNLP '09 248, 248 (2009), available athttp://dl.acm.org/ft_gateway.cfm?id=1699543&type=pdf&CFID=610241880&CFTOKEN=34401448. By contrast, discriminativeLDA provides “a supervised dimensionality reduction algorithm thatuncovers the latent structure in a document collection while preservingpredictive power for the task of classification.” S. Lacoste-Julien etal., DiscLDA: Discriminative Learning for Dimensionality Reduction andClassification, 21 NIPS08 (2008), available athttp://papers.nips.cc/paper/3599-disclda-discriminative-learning-for-dimensionality-reduction-and-classification.pdf.Engineers and computer scientists continue to face challenges in thefield of LDA variants.

SUMMARY

A computer-implemented method includes creating a classifier by:training a machine learning model using two or more tasks, wherein thetwo or more tasks lie in two or more domains; including in the machinelearning model at least one attribute common to at least two of said twoor more domains; including in the machine learning model at least onelatent feature that affects at least two of the two or more tasks thatfall within one of the at least two domains; and constructing theclassifier based on said machine learning model. Thecomputer-implemented method further includes applying the classifier toat least one operational task.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating a various elements ofa computing environment suitable for operating a classifier generationprogram, in accordance with at least one embodiment of the presentinvention.

FIG. 2 is a flowchart diagram depicting various operational steps for aclassifier generation program, in accordance with at least oneembodiment of the present invention.

FIG. 3 is a flowchart diagram depicting various operational steps for aclassifier generation program in the context of a cognitive servicessuite, in accordance with at least one embodiment of the invention.

FIG. 4A is a logical diagram of a learning model associated with LDAtechniques, in accordance with at least the modeled embodiment describedherein.

FIG. 4B is a logical diagram of a learning model associated with labeledLDA techniques, in accordance with at least the modeled embodimentdescribed herein.

FIG. 4C is a logical diagram of a learning model associated withdiscriminative LDA, in accordance with at least the modeled embodimentdescribed herein.

FIG. 5 is a logical diagram of a learning model incorporating bothlabeled LDA and discriminative LDA techniques, in accordance with atleast the modeled embodiment described herein.

FIG. 6A is a network diagram of a learning model associated with LDAtechniques, in accordance with at least the modeled embodiment describedherein.

FIG. 6B is a network diagram of a learning model associated with labeledLDA techniques, in accordance with at least the modeled embodimentdescribed herein.

FIG. 6C is a network diagram of a learning model associated withdiscriminative LDA, in accordance with at least the modeled embodimentdescribed herein.

FIG. 6D is a network diagram of a learning model incorporating bothlabeled LDA and discriminative LDA techniques, in accordance with atleast the modeled embodiment described herein.

FIG. 7A is a network diagram depicting a high level algorithmic overviewfor domain adaptation, in accordance with at least one embodiment of thepresent invention.

FIG. 7B is a network diagram depicting a high level algorithmic overviewfor multi-task learning, in accordance with at least one embodiment ofthe present invention.

FIG. 8 is a network diagram depicting a high level algorithmic overviewfor a classifier generation program incorporating both labeled LDA anddiscriminative LDA, in accordance with at least one embodiment of thepresent invention.

FIG. 9 is a block diagram depicting various logical elements for acomputer system capable of executing program instructions, in accordancewith at least one embodiment of the present invention.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of various components of an operationalenvironment for a classifier generation program 101, in accordance withat least one embodiment of the invention. A computer system 100 mayinclude any type of computing platform, including a single sever orworkstation computer, a distributed or virtual computer, or a portableor embedded device. In the depicted embodiment, the classifiergeneration program 101 produces and/or interacts with a classifier 102,which in turn operates based on a machine learning model 104.

The classifier 102 may be provided to one or more cognitive services192, which in turn may be aggregated into a cognitive services suite190. IBM® Watson® is one non-limiting example of a cognitive servicessuite wherein various cognitive services are made available via a set ofAPIs. In the depicted embodiment, the cognitive services 192 are madeavailable to user-level applications via an application programminginterface (API) 194. The API 194 may be accessed via a developmentplatform 196. In the depicted embodiment, the development platform 196may be a cloud-based development environment such as IBM® Bluemix®.Ultimately, an operational task 198 poses a real-world problem to theclassifier 102. In the depicted embodiment, the operational task 198arises from a user-level application that accesses one of the cognitiveservices 192. In alternative embodiments, the operational task mayinclude a user-level application that includes the classifier 102 in itsown executable, module, or other program package. In other alternativeembodiments, the operational task may access the classifier 102 via aclient/server or other network model, without the intermediary of acognitive services suite, such as the cognitive services suite 190.

Referring still to the embodiment depicted in FIG. 1, the machinelearning model 104 upon which the classifier 102 is based interacts inboth training and in new/operational exposure with various tasks anddomains. Examples of tasks include sentiment analysis (i.e., expressionsof positive or negative feelings when communicating on a topic) andfacial expression analysis (i.e., inferred emotional state by facialexpressions in conjunction with communicating on a topic). Where theseare applied to a video of a person speaking and expressing an opinion,such as whether the person likes or dislikes a food item or a famousperson, then the tasks may be understood as in one domain or the other.That is, if the video content is about a famous person or organization,notable persons may be understood to be the domain, and, if bothsentiment analysis and facial expression analysis are applied, then bothof those tasks may be understood as being in the domain of politics.Similarly, where both sentiment analysis and facial recognition areapplied to food preferences, then those same tasks may be understood asin the domain of food.

Thus, in the embodiment depicted in FIG. 1, the machine learning modelmay perform tasks and/or operate on attributes in both any previouslytrained domain 130 and any new domain 140. Shared between the previouslytrained domain 130 and the new domain 140 are various common attributes150. Similarly, the previously trained domain 130 may have one or morelatent features 134, which distinguish the previously trained domain 130from the new domain 140, which has its own latent features 144 that makea parallel distinction. Thus, by discovering which attributes of thepreviously trained domain 130 are common attributes 150 and which arelatent features 134, the classifier generation program 101 can determinewhich features are relevant to the classifier 102 for the new domain140, and domain adaptation is achieved.

Correspondingly, any previously trained domain 130 may include a newtask 132, and any new domain 140 may include a new task 142. The newtask 132 may be performed by the machine learning model 104 using all ofthe features of the previously trained domain 130 to achieve transferlearning. For the new task 142, which is in the new domain 140, theclassifier generation program 101 can generate the machine learningmodel 104 from the common attributes 150 and latent features 144, whichdefine the new domain 140. Thus, for the new task 142, the classifiergeneration program can achieve both transfer learning (new task) anddomain adaptation (new domain) in the same transition. The new task 142and/or the new domain 140 and/or the new task 132, when performed usingthe machine learning model 104, provide the power to discover at leastone unobserved group 180, which includes at least one previously unknowntopic 182.

Contrasting the computing environment model of FIG. 1 to humanintelligence, the inventors have observed that humans have the abilityto inherently transfer knowledge between tasks. Thus, the inventors havereasoned, human learners not only recognize but also apply relevantknowledge successfully to new tasks that require reflection andthoughtful application of knowledge gained from previous learningexperiences. By contrast, the inventors have observed and/or recognizedthat existing machine learning algorithmic paradigms focus on addressinglearning tasks in isolation where learning in one task is independent ofthe other. Consequentially, the inventors have reasoned, classifierapplications that are challenged by scarcity of labeled data becomedemanding problems for data scientists because the predictivecapabilities of machine learning algorithms are hindered in such cases.Moreover, the inventors have further reasoned, given that thedistribution of data changes over time, rendering models trained witholder data makes machine learning frameworks trained on older data muchless capable of discovering useful structures from newly available data.

The inventors have further observed and/or recognized that transferlearning thus emerges as a convenient concept to change theaforementioned notion of isolated learning by developing methods totransfer knowledge gained in one or more source tasks and exploit thelearned knowledge a related target task with improved learning. However,the inventors have further observed and/or recognized, transfer learningas a known framework works at identifying and overcoming target tasks byeither simultaneously training models across domains or by focusing on asequential transfer of knowledge from one domain to another. Theinventors have further observed and/or recognized that this is a currentsignificant limitation of transfer learning that restricts theapplication of transfer learning when the focus is on domain adaptationtasks that require the source and target tasks to be learnedsimultaneously through multi-task learning.

And so, the inventors have reasoned, while transfer learning is asuitable framework to overcome problems of isolated learning by learninga model specific to a domain and then to benefit from learning othermodels in other domains. This, the inventors have further observedand/or recognized, merely starts to scratch the surface for domainadaptation tasks that require multi-task learning because there is noreal attempt to allow for both simultaneous and sequential knowledgetransfer in an aggregated manner.

The inventors have further observed and/or recognized that one possiblepoint of novelty in the proposed system of the present invention restsin the framework's unique ability not only to share knowledge acrossmultiple tasks within related domains (for simultaneous learning), butalso to adapt learned distributions to fit data from new domains (forsequential learning). Thus, embodiments of the present inventionconceptualize a latent variable model and introduce two new frameworks:a modified supervised LDA model and its non-parametric variation.Embodiments of the present invention are thus capable, the inventorshave further observed and/or recognized, of integrating two differenttypes of supervision: topic labels and category labels, which rendersembodiments of the present invention applicable to multi-task learningendeavors whose primary goal is domain adaptation. Since both the latentand supervised topics are shared across multiple categories, embodimentsof the present invention incorporate the aforementioned uniquecharacteristics of the data.

Thus, the inventors categorize existing methodologies for knowledgeacquisition into two fields. The first field, inspired from humanperception and learning, observes that humans are capable of recognitionand inference through intrinsic correlation across various domainsbecause humans exhibit simultaneous knowledge learning. In the firstfield, human abilities are mimicked by mining features from data forwhich the distribution and feature space changes over time. The secondfield, derived from training tasks in isolation where each task islearned independently by a different classifier, is restricted to datawhose feature space and distribution is constant over time. Theinventors suggest that some embodiments of the present invention combineboth these fields and/or approaches to knowledge acquisition forbuilding a multi-task learning framework for domain adaptation.

Thus, in embodiments of the present invention, not only is theclassifier first trained to predict attributes from raw features andthen trained to predict categories from the attributes, but also theclassifier is capable of working efficiently with labeled data that ispresented via class labels. The inventors have further observed and/orrecognized that the aforementioned combined properties permit theclassifier to simultaneously acquire knowledge and also string theknowledge coherently into a sequence, which provides advancement in theart of machine learning. Further, the inventors have observed and/orrecognized that embodiments of the present invention advance the art byusing probabilistic latent variable models as the building blocks foraggregated knowledge acquisition. Further, the inventors have observedand/or recognized that the combined framework includes a capacity towork successfully with supervision that is naturally available fromdifferent sources at different levels of abstraction. For example,supervision may be provided as keywords, topics, and categories fordocuments in conjunction with visual attributes, objects, and scenelabels for images, and various embodiments would exhibit learn andtransfer knowledge across both documents and images, which may not onlyresult in improved internal representation of the document and/or imagedata, but also acquire, for the framework/model, an enhancedunderstanding of different tasks in different domains. In suchembodiments, higher/improved classification accuracy may be achievedbecause the classifier can infer object attributes by only witnessingraw object features on testing. Thus, the inventors conclude thatembodiments of the present invention uniquely exploit transfer learningfor tasks that require multi-task learning across domains.

The inventors have further observed and/or recognized that embodimentsof the present invention may have application at least in text analysis,object recognition from images, and recommender systems. Further, theinventors have observed and/or recognized that embodiments of thepresent invention have the ability to heavily discount the computationaleffort and/or load that is needed to recollect training data and rebuildmodels in cognitive services suites especially as such systemscapabilities are harnessed across varied tasks in varied domains.Further, the inventors have observed and/or recognized that embodimentsof the invention provide a framework that is capable of working fromhigh-level object descriptions, for example where a classifier ifapplied to an object recognition problem, that classifier would be ableto correlate a tripod as a stand with 3 legs. Further, the inventorshave observed and/or recognized that a classifier according to variousembodiments may be capable of being trained to examine therepresentations learned from previous tasks and use the knowledge thathas been gained independently in other tasks to make classificationsacross domains. It will be understood, however, that the aforementionedstatements, observations and/or recognitions of possible advantages ofthe present invention and/or problems with the prior art need notnecessarily be present and/or resolved in any particular embodiment ofthe present invention and are not to be understood as necessary for thepractice of the present invention.

Referring now to FIG. 2, FIG. 2 is a flowchart diagram for a classifiergeneration program 101, in accordance with at least one embodiment ofthe present invention. In some embodiments, the classifier generationprogram may be understood as a method for a multi-task learningframework to facilitate knowledge acquisition and transfer learningacross domains. In the depicted embodiment, at step 200, the classifiergeneration program trains a machine learning model, such as the machinelearning model 104, using two or more tasks. The two or more tasks liein two or more domains. The classifier generation program 101, at step202, includes in the machine learning model, such as the machinelearning model 104, at least one attribute common to at least two of thetwo or more domains. At step 204, the classifier generation program 101includes in the machine learning model, such as the machine learningmodel 104, at least one latent feature that affects at least two of thetwo or more tasks that fall within on the at least two domains.

Referring still to the embodiment depicted in FIG. 2, LDA is the basicbuilding block for performing the transfer learning and domainadaptation that underlies steps 200-204. LDA uses a generative modelthat allows sets of observations to be explained by unobserved groups ina manner that validates why some parts of a given input data aresimilar. Conventional LDA focuses on unsupervised induction of multiple“topics” and thus provides characterization for a corpus of textdocuments. FIG. 6A illustrates conventional LDA in terms of features600, which include all features of a given domain, latent features 610,which are those features that serve to distinguish a given topic fromother topics in the same space, corpus, or system. In FIG. 6A, time,designated t, is shown to move from left to right, wherein features 600begin as all latent, and then as the model is refined over time andsimilarities and differences between topics are learned, fewer featuresare identified as latent in the machine learning model, such as themachine learning model 104. In performing the training of steps 200-204,the LDA approach is augmented to include two different types ofsupervision.

Labeled LDA (Supervision type 1) includes the assessment of documentlevel labels for a topic. That is, supervision is provided by labelingeach document with its list of topics. For example, when rendering awebpage that includes tags, labeled LDA can be applied to the web pagecontent to highlight interesting portions of the content, based on thetags. Further, labeled LDA can also be applied to build a summary oftext from multiple webpages that share the same set of tags. Thus, thissupervision accounts for sequential learning akin to human perception.In terms of the embodiment depicted in FIG. 2, the classifier generationprogram 101 including in the machine learning model, at step 202, atleast one attribute common to at least two of said two or more domainscan include applying supervised machine learning, and the supervisedmachine learning can included labeled LDA.

FIG. 6B distinguishes labeled LDA from conventional LDA, as shown inFIG. 6A. In the diagram of FIG. 6B, time, t, progresses from left toright, and attributes 620, as opposed to latent features 610, arediscovered. Attributes 620, by contrast to latent features 610, showsimilarities and/or shared characteristics between topics and/or labels.Thus, if a group of features 600 are taken as to be within the samedomain, then by extraction of attributes 620 from those features, theattributes 620 may be understood to be attributes of the domain.

FIG. 7A places labeled LDA in context. The diagram of FIG. 7A depicts ahigh level algorithmic overview for domain adaptation using labeled LDA.The depicted diagram provides for sequential knowledge transfer akin tohuman perception wherein the same task is performed with multipledomains being factored in. In FIG. 7A, each set of shapes may beunderstood as a domain, with some task to be applied to those shapesthat is the same in each domain. As shown, the learner (i.e., a machinelearning model such as the machine learning model 104), is trained on asample of circles. Sequentially, the learner is exposed to thetriangles, and then, when the learner is applied to the rhombuses, thelearner can apply its gained knowledge from the circles and triangles toperform the same task for rhombuses.

Referring still to the embodiment depicted in FIG. 2, discriminative LDA(Supervision type 2) includes creating, by the classifier generationprogram 101, one or more overall category inferences from varioustopics. Discriminative LDA may be understood to provide supervisionthrough a single response variable, for which the task is to predict thevalue of the response variable for a given document. In variousembodiments employing discriminative LDA, the response can be a realvalue or a categorical value or identifier. In various embodimentsemploying discriminative LDA, the response can be modeled by a normal,Poisson, binomial, or any similar distribution. For example, where theobjective is to infer a low dimensional, topic based representation ofone or more documents, embodiments of the classifier generation program101 employing discriminative LDA can both perform dimensionalityreduction and provide improved classification in the dimensionallyreduced space through more trivial update equations.

Thus, discriminative LDA can account for simultaneous learning, akin toconventional machine learning algorithms wherein multiple tasks in thesame domain can factor into the machine learning model. By contrast tolabeled LDA, which only uses supervised topics and does not have anymechanism for generating the class labels (labeled LDA may be understoodto correspond to and/or approximate the function of attributes in acomputer vision task), discriminative LDA only has latent topics and/orfeature (as in step 204) but learns a discriminative model forpredicting classes from these topics (discriminative LDA may beunderstood to correspond to and/or approximate in function the hiddenlayer in conventional multi-task learning, where there is no supervisionin the form of topic tags or categories assigned to a document, thusdiscriminative LDA may be understood equivalently as both supervisedthrough the response variable and unsupervised because it lacks giventopics).

Further, FIG. 6C depicts a diagram for a machine learning modelemploying discriminative LDA. As shown, the features 600 identify,through the machine learning model, latent features 610 thatdiscriminate between topics. The latent features 610 in turn provideclassification for various tasks 630, which are discriminated betweenthrough the response variable. With reference to the diagram of FIG. 6C,embodiments of the classifier generation program 101 that invokediscriminative LDA may be understood to create an overall categoryinference that predicts classes based on the document topics (or theimage attributes, or the like) in a manner that transfers across tasksand does not involve supervision in the sense that labeled LDA issupervised.

FIG. 7B provides a high level algorithmic overview of discriminativeLDA, wherein, as per the left half of the diagram, raw input trains ashared intermediate input, from which specific classifiers (smallershaded ovals) are trained to provide output for various tasks. On theright half of the diagram, various output classifications are shown toprogress from distinguishing lower level features to higher levelfeatures, with the higher level features being progressively moreabstract and finely tuned to more subtle patterns.

In terms of the embodiment depicted in FIG. 2, the classifier generationprogram 101, at step 204, may be understood to include in the machinelearning model at least one latent feature that affects at least two ofthe two or more tasks that fall within one of the at least two domainsby applying unsupervised machine learning, which includes discriminativeLDA.

FIG. 8 provides a high level algorithmic view of how both labeled anddiscriminative LDA can apply in a single machine learning framework, andhow a classifier based on the combined model is operable to applyderived information both concurrently (discriminative LDA) andsequentially (labeled LDA). Embodiments of the invention conceptualizethe above-discussed concepts as a new modified supervised LDA frameworkand its non-parametric variation. Thus, if a classifier according tovarious embodiments of the invention is working with a training corpusincluding N documents which belong to Y different classes, where eachdocument belongs to exactly one class and each class corresponds to adifferent task, such a classifier may train a machine learning modelusing words in the data as well as associated supervised topic tags andclass labels, which may be provided through either text or images, andthen use the machine learning model to classify test data that iscompletely unlabeled with no topic tags nor class labels. FIG. 6Dprovides a diagrammatic explanation of a combined LDA framework, inaccordance with an embodiments wherein a combined layer 650 of latentfeatures 610 and attributes 620 makes up the intermediate knowledge of amachine learning model from which various classifiers can be generatedfor various tasks 630.

Thus, referring still to the embodiment of FIG. 2, at step 206, theclassifier generation program 101 constructs, creates, or instantiates aclassifier, such as the classifier 102, based on the machine learningmodel, such as the machine learning model 104. As generally understoodin the field of machine learning, the term “classifier”, including theclassifier 102, means a concrete implementation of an algorithm thatimplements classification. LDA, including labeled LDA, discriminativeLDA, and combined LDA according to various embodiments of the presentinvention, may be understood to be examples of classificationalgorithms. Thus, “classifier” as used herein means any softwarefunction, method, object, module, engine, etc. that performsclassification according to a model, such as the machine learning modelsherein described. A machine learning model itself, such as the machinelearning model 104, may be understood as any electronic representationthat maps input data to machine learning decisions, including LDAfunctions as described herein, neural nets, etc. For a machine learningmodel to be understood to include some data, such as having been trainedon a given corpus, it will be understood that such inclusion means thatthe corpus has affected the machine learning module in some way suchthat the operation of the machine learning model necessarily is takingthe corpus into account without necessarily providing ahuman-predictable result because of such inclusion.

Referring still to the embodiment of FIG. 2, at step 208, the classifiergeneration program 101 applies the classifier, such as the classifier102, to at least one operational task. The operational task may beunderstood as a real-world classification task with real data, asopposed to training data with known results. With reference to FIG. 1,the operational task can be at least one of: (a) a new task in apreviously trained domain (transfer learning); (b) a previously trainedtask in a new domain (e.g., previously trained task 143) (domainadaptation); and (c) a new task in a new domain (simultaneous transferlearning and domain adaptation). In various embodiments, the classifier,such as the classifier 102, can be understood to provide a latentgenerative model that includes sets of observations of one or moretopics explained by one or more unobserved groups in at least one of (a)one or more of the two or more tasks; and (b) one or more of the two ormore domains; and wherein the latent generative model is applied to theoperational task.

Referring now to FIG. 3, FIG. 3 is a flowchart diagram for a classifiergeneration program 101, according to at least one embodiment of theinvention. In the embodiment depicted in FIG. 3, at step 300, theclassifier generation program 101 identifies a cognitive services suite,such as the cognitive services suite 190. The cognitive services suite190 includes one or more cognitive services 192. Examples of cognitiveservices include: (a) natural language classification; (b) sentimentanalysis; (c) retrieval and ranking; (d) concept insight generation; (e)natural language processing; (f) natural language translation; (g) toneanalysis; (h) personality insight generation; (i) conversational dialog;(j) entity and relationship extraction; (k) customer engagementanalysis; and (l) relationship discovery (e.g., in the life sciences,drug discovery, and oncological diagnosis applications). Examples ofunderlying operational tasks of various cognitive services 192 caninclude sentiment analysis, retrieve and rank, discovery, decision,policy, etc.

Referring still to the embodiment of FIG. 3, at step 302, analogously tostep 202, the classifier generation program 101 creates a classifier bytraining a machine learning model, such as the machine learning model104, using two or more tasks, wherein the two or more tasks lie in twoor more domains. At step 304, analogously to step 204, the classifiergeneration program 101 includes in the machine learning model, such asthe machine learning model 104, at least one attribute common to atleast two of the two or more domains. At step 306, analogously to set206, the classifier generation program 101 includes in the machinelearning model, such as the machine learning model 102, at least onelatent feature that affects at least two of the two or more tasks thatfall within one of the at least two domains.

Referring still to the embodiment depicted in FIG. 3, at step 308, theclassifier generation program 101 constructs the classifier, such as theclassifier 102, based on the machine learning model, such as the machinelearning model 104, and incorporates the classifier into at least of theplurality of cognitive services, such as the cognitive services 192. Atstep 310, the classifier generation program incorporates the classifier,such as the classifier 102, into at least two of the plurality ofcognitive services 192. Thus, for the cognitive services suite 190, thebenefits of combined transfer learning and domain adaptation as well ascombined sequential and simultaneous learning are incorporated into thecognitive service suite 190 at a fundamental level such that at leasttwo of the cognitive services 192 (analogous to operational tasks)benefit from the enhanced learning.

Thus, in some embodiments at least two of the plurality of cognitiveservices include distinct operational tasks within a single domain, forexample where the two services are applied to similar topic areas, forexample applying both sentiment analysis (an operational task) andfacial expression analysis (another operational task) to videos ofpeople describing how they feel about food (a domain). Similarly, insome embodiments, at least two of the plurality of cognitive servicescomprise a common operational task within distinct operational domains(as per FIG. 4A). In a similar example, one cognitive service may applysentiment analysis (a task) to opinions about food (a domain), andanother cognitive service may apply sentiment analysis (the same task)to opinions about politicians (another domain, for example in FIG. 4C).In other embodiments, at least two of the plurality of cognitiveservices comprise distinct operational tasks in distinct operationaldomains.

Further, the cognitive services suite 190 may be accessible through atleast one API 194, and the API may be integrated with and/or accessiblevia a cloud-based application development platform, such as thedevelopment platform 196. The development platform 196 can include anysystem that provides any degree of infrastructure for cloud-basedapplications together with, in the same computing environment, an APIfor a cognitive service 192. Examples of cloud-based infrastructureinclude software-as-a-service, infrastructure-as-a-service, andframework-as-a-service models.

A theoretical validation of the proposed system has been presentedbelow, with reference to FIGS. 4A, 4B, 4C, and 5. The proof recites thegenerative model of modified supervised LDA according to variousembodiments of the invention as well as provides backing on how theinference and learning occurs across different tasks and differentdomains.

Referring now to a generative model, the inventors propose amathematically described model that demonstrates one embodiment of theinvention. The modeled embodiment, depicted symbolically in FIGS. 4A,4B, 4C, and 5, addresses a task of topic selection over a plurality ofdocuments. It will be understood that the recitation of the model forthe modeled embodiment is not to be construed as limiting on or requiredfor all embodiments that practice the invention.

With respect to the modeled embodiment, for the n^(th) document, let thetopic selection probability, as per FIG. 4B, vector be defined asfollows: D_(n)˜Dir(α_(n)) where α_(n)=

_(n)·α; and α is a parameter of Dirichlet distribution of dimension K;and K is the total number of topics. Of the topics, latent topics

K₁, and supervised topics

K₂.

_(n) is a diagonal binary matrix of dimension K×K.

Referring still to the modeled embodiment, considering a feature vectorwherein the Kth diagonal entry will be 1, in the cases shown:

$\begin{matrix}{{{The}\mspace{14mu} K^{th}\mspace{14mu}{diagonal}\mspace{14mu}{entry}} = \begin{Bmatrix}1 & {1 \leq K \leq K_{1}} \\{or} & {K_{1} \leq K \leq K} \\{and} & {n^{th}\mspace{14mu}{doc}\mspace{14mu}{tagged}\mspace{14mu} K^{th}\mspace{14mu}{topic}}\end{Bmatrix}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

Also, let α=(α₁,α₂), where α₁ is a parameter of a Dirichlet distributionof dimension K₁ and α₂ is a parameter of a Dirichlet distribution ofdimension K₂. For the m^(th) word in the n^(th) document, sample atopic: Z_(nm)·multinomial (θ_(n)′) where:θ_(n)′=(1−ε){θ_(nK)}_(K=1) ^(K) ¹ +ε{

_(n,kk)θ_(nK)}_(K=1+K) ₁ ^(K)   Equation 2

In Equation 2, with reference to FIG. 5, ε is a term to weight latentvs. supervised topics. Now sample word W_(mn)˜multinomial (β_(z) _(k,m)) where β_(k) is a multinomial distribution over a vocabulary of wordscorresponding to the K^(th) topic. For the n^(th) document, generate:

$y_{n} = {\sum\limits_{m = 1}^{m_{n}}\frac{Z_{nm}}{M_{n}}}$arg max_(y) γ_(y) ^(T)E(Z_(n) ) where y_(n) in a class label associatedwith the n^(th) document and where Z_(nm) is an indicator vector ofdimension K and γ_(y) is a K-dimensional real vector corresponding tothe y^(th) class. M_(n) is the number of words in the n^(th) document.γ_(y) is computed using the max-margin principle.

Referring still to the generative model, with respect to inference andlearning, let one or more hidden variables Z, and one or more observedvariables X be given as follows:Z={{z _(nm)},{θ_(n)}}  Equation 3AX={w _(nm)}  Equation 3B

For a model parameter k₀, a joint distribution of the hidden andobserved variables N may be expressed:

$\begin{matrix}{{p\left( {X,\left. 2 \middle| K_{0} \right.} \right)} = {\prod\limits_{n = 1}^{N}{{p\left( \theta_{n} \middle| \alpha_{n} \right)} \cdot {\prod\limits_{m = 1}^{M_{n}}{{p\left( Z_{nm} \middle| \theta_{n}^{\prime} \right)} \cdot {p\left( w_{n,m} \middle| \beta_{z_{n,m}} \right)}}}}}} & {{Equation}\mspace{14mu} 4A}\end{matrix}$

To avoid computational intractability, a factorized approximation may beused, which is given by:

$\begin{matrix}{{q\left( Z \middle| \left\{ K_{n} \right\}_{n = 1}^{N} \right)} = {\prod\limits_{n = 1}^{N}{{q\left( \theta_{n} \middle| \gamma_{n} \right)} \cdot {\prod\limits_{m = 1}^{M}{q\left( Z_{nm} \middle| \phi_{nm} \right)}}}}} & {{Equation}\mspace{14mu} 4B}\end{matrix}$Where:θ_(n) ˜Dir(γ_(n))∀n={1, . . . ,N}   Equation 4CZ _(nm)˜multinomial(ϕ_(nm))∀n={1, . . . N} and m={1, . . . ,N}  Equation 4DK _(n)={γ_(n),{ϕ_(nm)}}   Equation 4EAdditionally:γ_(n)=(γ_(nk))_(k=1) ^(k) ∀n   Equation 5ϕ_(nm)=(ϕ_(nmk))_(k=1) ^(k) ∀n,m   Equation 6

Thus, the modified supervised LDA model for various embodiments reducesto the optimization problem:

$\begin{matrix}{\min\left( {q,K_{0},{{{\left\{ ɛ_{n} \right\} \cdot \frac{1}{2}}{r}^{2}} - {L\left( {q(Z)} \right)} + {C{\sum\limits_{n = 1}^{N}ɛ_{n}}}}} \right.} & {{Equation}\mspace{14mu} 7}\end{matrix}$Such that:∀n,y≠y _(n) :E[γ^(T) ·Δf _(n)(y)]≥1−ε_(n);ε_(n)≥0   Equation 8

In the model presented herein, the slack variables are denoted asfollows:

$\begin{matrix}\begin{bmatrix}{{\Delta\;{f_{n}(y)}_{N}} = {f\left( {y_{n},\overset{\_}{z_{n}}} \right)}} & {- {f\left( {y,\overset{\_}{z_{n}}} \right)}} \\\left\{ ɛ_{n} \right\}_{n = 1} & \;\end{bmatrix} & {{Equation}\mspace{14mu} 9}\end{matrix}$

The modified supervised LDA model projects the documents onto a combinedtopic space and uses a max-margin principle to predict class labels:

$\begin{matrix}{{f\left( {y,{\overset{\_}{z}}_{n}} \right)} = \left\{ \begin{matrix}{components} & {{of}\mspace{14mu}\overset{\_}{z_{n}}\mspace{14mu}{from}} \\{{\left( {y - 1} \right)k} + 1} & {{to}\mspace{14mu}{yk}} \\0 & {else}\end{matrix} \right.} & {{Equation}\mspace{14mu} 10}\end{matrix}$

Expectation Step (E-Step):

${{\phi_{nmk}^{*} \propto} ⩓_{n,{kk}}{{\exp\left\lbrack {{\Psi\left( Y_{n_{k}} \right)} + {\log\left( {\beta_{k}w_{nm}} \right)} + {\log\left( e^{\prime} \right)} + {\frac{1}{M_{n}}{\sum\limits_{y \neq y_{n}}{{M_{n}(y)}{E\left\lbrack {{\gamma\; y_{n}k} - \gamma_{yk}} \right\rbrack}}}}} \right\rbrack}{\forall n}}},m,{k.}$Thus:

$\begin{matrix}{\mspace{79mu}{{\gamma_{nk}^{*} = {⩓_{n,{kk}}\left\lbrack {\alpha_{k} + {\sum\limits_{m = 1}^{M_{n}}\phi_{nmk}}} \right\rbrack}},{\forall{n.m}},k}} & {{Equation}\mspace{14mu} 11} \\{\mspace{79mu}{{B_{kv}^{*} \propto {\sum\limits_{n = 1}^{N}{\sum\limits_{m = 1}^{M_{n}}{{\phi_{nmk} \cdot {I\left( {w_{nm} = v} \right)}}{\forall n}}}}},k,v}} & {{Equation}\mspace{14mu} 12} \\{L_{(\frac{\alpha_{1}}{\alpha_{2}})} = {\left\lbrack {{\sum\limits_{n = 1}^{N}{\log\left( {r\left( {\sum\limits_{k = 1}^{k}\alpha_{nk}} \right)} \right)}} - {\sum\limits_{n = 1}^{N}\;{\sum\limits_{k = 1}^{K}{\log\left( {r\left( \alpha_{nk} \right)} \right)}}}} \right\rbrack + {\sum\limits_{n = 1}^{N}\;{\sum\limits_{k = 1}^{K}{\left\lbrack {{\Psi\left( \gamma_{nk} \right)} - {\Psi\left( {\sum\limits_{k = 1}^{k}\gamma_{nk}} \right)}} \right\rbrack\left( \alpha_{nk} \right)}}}}} & {{Equation}\mspace{14mu} 13}\end{matrix}$

Maximization step (M-Step): Maximize objective function with respect tok₀, and update values for α₁, α₂ using support vector machine solver.Optimize the objective function with respect to γ. Thus γ can beoptimized, and therefore, the inventors have concluded that theframework can be reduced to an optimization problem, making it possibleto make predictions not only across tasks, but also across domains.

FIG. 9 is a block diagram depicting components of a computer 900suitable for executing the classifier generation program 101. FIG. 9displays the computer 900, the one or more processor(s) 904 (includingone or more computer processors), the communications fabric 902, thememory 906, the RAM, the cache 916, the persistent storage 908, thecommunications unit 910, the I/O interfaces 912, the display 920, andthe external devices 918. It should be appreciated that FIG. 9 providesonly an illustration of one embodiment and does not imply anylimitations with regard to the environments in which differentembodiments may be implemented. Many modifications to the depictedenvironment may be made.

As depicted, the computer 900 operates over a communications fabric 902,which provides communications between the cache 916, the computerprocessor(s) 904, the memory 906, the persistent storage 908, thecommunications unit 910, and the input/output (I/O) interface(s) 912.The communications fabric 902 may be implemented with any architecturesuitable for passing data and/or control information between theprocessors 904 (e.g., microprocessors, communications processors, andnetwork processors, etc.), the memory 906, the external devices 918, andany other hardware components within a system. For example, thecommunications fabric 902 may be implemented with one or more buses or acrossbar switch.

The memory 906 and persistent storage 908 are computer readable storagemedia. In the depicted embodiment, the memory 906 includes a randomaccess memory (RAM). In general, the memory 906 may include any suitablevolatile or non-volatile implementations of one or more computerreadable storage media. The cache 916 is a fast memory that enhances theperformance of computer processor(s) 904 by holding recently accesseddata, and data near accessed data, from memory 906.

Program instructions for the classifier generation program 101 may bestored in the persistent storage 908 or in memory 906, or moregenerally, any computer readable storage media, for execution by one ormore of the respective computer processors 904 via the cache 916. Thepersistent storage 908 may include a magnetic hard disk drive.Alternatively, or in addition to a magnetic hard disk drive, thepersistent storage 908 may include, a solid state hard disk drive, asemiconductor storage device, read-only memory (ROM), electronicallyerasable programmable read-only memory (EEPROM), flash memory, or anyother computer readable storage media that is capable of storing programinstructions or digital information.

The media used by the persistent storage 908 may also be removable. Forexample, a removable hard drive may be used for persistent storage 908.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is also part of the persistentstorage 908.

The communications unit 910, in these examples, provides forcommunications with other data processing systems or devices. In theseexamples, the communications unit 910 may include one or more networkinterface cards. The communications unit 910 may provide communicationsthrough the use of either or both physical and wireless communicationslinks. Classifier generation program 101 may be downloaded to thepersistent storage 908 through the communications unit 910. In thecontext of some embodiments of the present invention, the source of thevarious input data may be physically remote to the computer 900 suchthat the input data may be received and the output similarly transmittedvia the communications unit 910.

The I/O interface(s) 912 allows for input and output of data with otherdevices that may operate in conjunction with the computer 900. Forexample, the I/O interface 912 may provide a connection to the externaldevices 918, which may include a keyboard, keypad, a touch screen,and/or some other suitable input devices. External devices 918 may alsoinclude portable computer readable storage media, for example, thumbdrives, portable optical or magnetic disks, and memory cards. Softwareand data used to practice embodiments of the present invention may bestored on such portable computer readable storage media and may beloaded onto the persistent storage 908 via the I/O interface(s) 912. TheI/O interface(s) 912 may similarly connect to a display 920. The display920 provides a mechanism to display data to a user and may be, forexample, a computer monitor.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A computer-implemented method comprising:identifying a cognitive service suite, said cognitive service suitecomprising a plurality of cognitive services; creating a classifier by:training a machine learning model using two or more tasks, wherein: saidtwo or more tasks lie in two or more domains; said machine learningmodel comprises a combination of both labeled latent Dirichletallocation and discriminative latent Dirichlet allocation; and saidlabeled latent Dirichlet allocation comprises a diagonal binary matrixof a dimension equal to total number of topics by total number oftopics; including in said machine learning model at least one attributecommon to at least two of said two or more domains, wherein saidattribute is a shared characteristic; including in said machine learningmodel at least one latent feature that affects at least two of said twoor more tasks that fall within one of said at least two domains, whereinsaid latent feature differentiates a topic within said one of said atleast two domains; and constructing said classifier based on saidmachine learning model; and incorporating said classifier into at leasttwo of said plurality of cognitive services, wherein a first cognitiveservice of said plurality of cognitive services comprises facialexpression analysis to infer emotional state in conjunction withcommunicating on a first topic.
 2. The computer-implemented method ofclaim 1, wherein said at least two of said plurality of cognitiveservices comprise distinct operational tasks within a single operationaldomain.
 3. The computer-implemented method of claim 1, wherein said atleast two of said plurality of cognitive services comprise a commonoperational task within distinct operational domains.
 4. Thecomputer-implemented method of claim 1, wherein said at least two ofsaid plurality of cognitive services comprise distinct operational taskswithin distinct operational domains.
 5. The computer-implemented methodof claim 1, wherein said plurality of cognitive services furthercomprises at least one service selected from the group consisting of:sentiment analysis; tone analysis; personality insight generation;conversational dialog; and customer engagement analysis.
 6. Thecomputer-implemented method of claim 1, wherein said cognitive servicesuite is accessible through at least one application programminginterface.
 7. The computer-implemented method of claim 6 wherein said atleast one application programming interface is integrated with acloud-based application development platform.
 8. Thecomputer-implemented method of claim 1, wherein said machine learningmodel further comprises weighting latent topics versus supervisedtopics.
 9. The computer-implemented method of claim 1, wherein saidlabeled latent Dirichlet allocation further comprises: generating aweight of a particular topic in a document by multiplying (i) aparameter of Dirichlet distribution of a dimension equal to the totalnumber of topics multiplied by (ii) the diagonal binary matrix.
 10. Acomputer program product comprising one or more computer readablestorage media and program instructions stored on said one or morecomputer readable storage media, said program instructions comprisinginstructions to: identify a cognitive service suite, said cognitiveservice suite comprising a plurality of cognitive services; create aclassifier by: training a machine learning model using two or moretasks, wherein: said two or more tasks lie in two or more domains; saidmachine learning model comprises a combination of both labeled latentDirichlet allocation and discriminative latent Dirichlet allocation; andsaid labeled latent Dirichlet allocation comprises a diagonal binarymatrix of a dimension equal to total number of topics by total number oftopics; including in said machine learning model at least one attributecommon to at least two of said two or more domains, wherein saidattribute is a shared characteristic; including in said machine learningmodel at least one latent feature that affects at least two of said twoor more tasks that fall within one of said at least two domains, whereinsaid latent feature differentiates a topic within said one of said atleast two domains; and constructing said classifier based on saidmachine learning model; and incorporate said classifier into at leasttwo of said plurality of cognitive services, wherein a first cognitiveservice of said plurality of cognitive services comprises facialexpression analysis to infer emotional state in conjunction withcommunicating on a first topic.
 11. The computer program product ofclaim 10, wherein said at least two of said plurality of cognitiveservices comprise distinct operational tasks within a single operationaldomain.
 12. The computer program product of claim 10, wherein said atleast two of said plurality of cognitive services comprise a commonoperational task within distinct operational domains.
 13. The computerprogram product of claim 10, wherein said at least two of said pluralityof cognitive services comprise distinct operational tasks withindistinct operational domains.
 14. The computer program product of claim10, wherein said plurality of cognitive services further comprises atleast one service selected from the group consisting of: sentimentanalysis; tone analysis; personality insight generation; conversationaldialog; and customer engagement analysis.
 15. A computer systemcomprising: one or more processors; one or more computer readablestorage media; computer program instructions; said computer programinstructions being stored on said one or more computer readable storagemedia; said computer program instructions comprising instructions to:identify a cognitive service suite, said cognitive service suitecomprising a plurality of cognitive services; create a classifier by:training a machine learning model using two or more tasks, wherein: saidtwo or more tasks lie in two or more domains; said machine learningmodel comprises a combination of both labeled latent Dirichletallocation and discriminative latent Dirichlet allocation; and saidlabeled latent Dirichlet allocation comprises a diagonal binary matrixof a dimension equal to total number of topics by total number oftopics; including in said machine learning model at least one attributecommon to at least two of said two or more domains, wherein saidattribute is a shared characteristic; including in said machine learningmodel at least one latent feature that affects at least two of said twoor more tasks that fall within one of said at least two domains, whereinsaid latent feature differentiates a topic within said one of said atleast two domains; and constructing said classifier based on saidmachine learning model; and incorporate said classifier into at leasttwo of said plurality of cognitive services, wherein a first cognitiveservice of said plurality of cognitive services comprises facialexpression analysis to infer emotional state in conjunction withcommunicating on a first topic.
 16. The computer system of claim 15,wherein said at least two of said plurality of cognitive servicescomprise distinct operational tasks within a single operational domain.17. The computer system of claim 15, wherein said at least two of saidplurality of cognitive services comprise a common operational taskwithin distinct operational domains.
 18. The computer system of claim15, wherein said at least two of said plurality of cognitive servicescomprise distinct operational tasks within distinct operational domains.19. The computer system of claim 15, wherein said plurality of cognitiveservices further comprises at least one service selected from the groupconsisting of: sentiment analysis; tone analysis; personality insightgeneration; conversational dialog; and customer engagement analysis. 20.The computer system of claim 15, wherein said cognitive service suite isaccessible through at least one application programming interface.